
Office 




INVESTOR IN FEOFUa 



The Patent Office 
Concept House 
Cardiff Road 
Newport. 



South Wales 

NPIO 8(#C'D2 8 JUN 2004 



WlPO 



POT 



I, the undersigned, being an officer duly authorised in accordance with Section 74(1) and (4) 
of the Deregulation & Contracting Out Act 1994, to sign and issue certificates on behalf of the 
Comptroller-General, hereby certify that annexed hereto is a true copy of the documents as 
origmally filed in connection with the patent application identified therein. 



In accordance with the Patents (Companies Re-registration) Rules 1982, if a company named 
in tois certificate and any accompanying documents has re-registered under the Companies Act 
lyso with the same name as that with which it was registered unmediately before re- 
registration save for the substitution as, or inclusion as, the last part of the name of the words 
public hm^ company" or their equivalents m Welsh, references to the name of the company 
in tills certificate and any accompanying documents shall be treated as references to the name 
with which It IS so re-registered. 

p^cTL^rlpuf ^ rules, the words "public limited company" may be replaced by p.l.c. 



Re-registration under the Companies Act does not constitute a new legal entity but merely 
™i^t^^ company to certain additional company law rules. 






Signed 
Dated 17 June 2004 



PRIORITY 
DOCUMENT 

SUBMirrED OR TRANSMITTED IN 
COMPLIANCE WITH RULE 17.1(a) OR (b) 



Patents Form 1/77 



Patents/" 
(Rulell. 



The 

Patent 



]STJ 



Request for grant of a patent 



f^: B % 



1. Your reference 




1/77 



■nie Patent Office 

Newport 
GwentNPlOSQQ 



A30349 



2. 



Patent application number 

Paent Ogice wUlfitt In this part) 

wiu nanie, address and postcode of the or of 



Patents ADP number &fyouk,u>wit) . 

If the applicant is a corporate body, give the 
-gga ^tate of its incorp nr.ri»« ^ ^ 
Title of the invention 



19JUHQ3 E816041--1 &0.3 052 

0'^i41«i -i i-Ji/ r'TOlj b.Uu t/oi^iolcl- 

' ^'^'- I _1 8 JUN 2003 

LONDON. EC1A7AJ. England 
Registered In England: 1 800000 



1867002-^ 
UNITED KINGDOM 



Name of your agent (^ymluwone) 

"t^t^^' ^ ^ Kingdom 
towh^ch aU correspondence should besL 
Vf^udmg the postcode) "^wau 



Patents ADP number (ifvouk^M 



EDGE ANALYSIS IN VIDEO QUALITY ASSESSMENT 



BT Group Legal 

fatellectual Property Department 
BT Centre 

8 1 Newgate Street C R 
I-ONDON EC1A7A. , 

/ 



ITMENT 



u you are declaring priority from one or more" 
rarher patent applications, give the country 

SS^^*^ and rCTj^K W ft) the or 
each apphcation number 

Ifttus a^Ucation is divided or otherwise 
denved from an earlier UK ^plication 
give the number and the filin^Le of 
me earlier appUcation 



1g6700f 

Countiy 



Priority application number 
Ttfyouknawit) 



Date of filing 
(4fay /month /year) 




b a^ement of mventorship and of right ' 
to grant of apatent required in supports 
^ request? t^^wgr 'Yes' ^: 
A| ^VpUcamfwmedinpanaismmtmemor or 

mnamedappUcmaisacotpormbody. 

(See note (<^) 

itents Form 1/77 



YES 



Enter the munber of sheets for any of the 
followmg items you are filing with this form. 
l^^ot count copies of the same document 

Continuation sheets of this form 

Description 37 

CIaini(s) 6 

Abstract 1 

Drawing(s) 18 

10. If you are also filing any of the following, 
state how may against each item 

* Priority Documents 

Translations of priority documents 

Statement of inventorship and ri^t 
to grant of apatKlt (PatentsForm 7/77) 

Request for preliminary examination YES 
and search (Patents Form 9/77) 

Request for substantive examination 
patents Form 10/77) 

Any other documents 



11. 



12. Name and daytime telephone number of 
person to contact in the United Kingdom 
Warning 



lAVe request the grant ofa patent on the basis of this application 
Signature(s) / / Date: 

WALLIN. Nicholas James, Authorised Signatory 



Rod Hiflen 



020 7492 8140 



P^^^ t^Z I """y- P'^^hermore, if you live in the United Kingdom Secdon ^cf ^e 

airectionprohibitmg publication or comnrnucation has been given, or any such direction has been revoked. 



Notes 
a) 

b) 

c) 



e) 
J) 



If you need help to fill in this form or you have any questions, please contact the Patent Office on 0645 500505. 
Write your answers in capital letters using black ink or you may type them. 

If mere is not enough space for all the relevant details on any part of this form, please continue on a separate sheet ^ 
pcverandwrite 'see continuation sheet" in the relevant part(s). Any continuation sheet should be attached to this form, 
^you have answered 'Yes' Patents Form 7/77 wiU need to be filed. 
Once you have filled in the form you must remember to dgn and date it. 
For dettdJs ofOiefee and ways to pay please contact the Patent Office. 



1/18 




2/18 




3/18 




4/18 




5/18 




6/18 



Q ZD 

1= 




o 


(£'0)D 


(O'O)D 


o" 

a 



<0 



1 

< 

< 


5' 


(e'o)D 


a 


a 


\ g 

\o 

\ 

\ 
\ 
\ 


a 


o j 


i 

t \ \ 

i ^ 



ix- 



7/18 




8/18 




9/18 



.1= 
S 



or 




a? 

O) 



Q 
UJ 

Q O 
Q 



Q O 
"J 2 O 




UJ o LL. 

O ^ UJ o 



10/18 



CO 



00 
CM* 





CO 




Q 

^° 
§2 

P- CD 
CO 







FORf 
■ PIXEL X 
X- 



LU 




CM 
D) 



CO 



11/18 





dNSd 



15/18 



a: 



I I 




16/18 



a: 2 

CO CL 




17/18 




18/18 



CO D. 




MS.doc 



35 



^^^^^^^^^^^^^^^^^^^ 

Technical FiftH 

Backarn..nH f» tK^ fnvrntinn and Prinr 
se,ec^d..es,ea,or.heir^sua,capa«es, ht™ ;~ 

30 minutes in a oo^^ environmen., and asked LT^ " h '° 
one of a varied of manners Usualiv f^I^ 7 °' '"^ ''=^"*» 

sequence under test with respect to ti^Jren-L " '"'"^ '° 

.easu^ments can found; «;::rrda~s;r^^^^^^^^^^ 
Su,^ Assessment of «» Quaiity „f Teievisil Pic, J ° '"f'°'^*»>"- 
issued in 1974 and Is fom,ally Known L CCIR 21 ? ' '"^ 

^0 cove..Hepas.p™posedme:ods,.rs^M^el:r """"''^^^ 

and i, can wo* we,, r^a^dl" "T svetems. 

.e. are Car J::;arrnrar :ir s:!r:::: ~- ■ 

5 Obtained, that me«culous set up and conL a" ^uC andtr T 
s.e.is«cai^ signiflcan. .^uits a g^t many huma II; !: ' T ? 
s^ened. These disadvantages render suhJecUv, testing compleTaL ,L ^ 
w* «,e resu,t that whilst subjective tests may be appiicableTo dev., 

. ^^'^rrerr;:ru:dTr7--~ 
— -e, .he^T: at rrr i rzrr: - 

assessment of video qualitv u.,inn * ^ ^''^ automatic 

Huaiiiy, using automated, and usuaih/ ^ 
comparison techniques A nri«r I . ^ computer based, video 

^nniques. A pnor art system whicli performs automof.v • . 
analysis is the PQA 300 system ftom TeKtronlx ,no of lCsW KaH p ^'^"" ''"^'"^ 
600, Beaverton, Oregon 97077 USA n,e P<M 300 w^rLTea? ' ° 

ouu worKs by measunng a two second 
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port<^ Of a Ave second video test sequence. The video test sequences may ,« 
downioaded ™, CD ROM or recorded ,rc. video, and piayed ou, .o he sysL " ndt 

™.h DSP acc^erated ha^ne on *e ^ second sequence. The measurement 
5 ,n a single numeric vaiue of piCu™ quaiity caiied the -picture quality rating- The 

emp.o>« a human vision system model known as MO MetHx and performs .ree ^eZ 
^pee Of analyse of the video infoonatton, being spatial analysis, temporal ana^tTd 
colour analysis, in o,.er to generBte the picture quality rating. Additionally Z PQA 
00 p^v^es PSNR values wh^ a. displayed in the form of an animated map whTse 
10 n.ene,.y ,s related to the PSNR differences ,«,„een .he reference and the test imtes 
in summary therefore, the PQA 300 is able to analyse test and referencT d" 
sequence. „ ord,r.o generate a video quafrty value, as wei, as PSNR measurements 

Problems can arise, however, with slraightfonvard comparisons of test and 
reference sequences to generate the quality metNcs mentioned above. Fo Ixlp e 
15 spafal or tempore, misaiignmen. between the whCe or parts of the referenL n^e 
test sequence can greatly afteot such measurements, but may be perceptu* 
,ns,gn.^n, to a human viewer. Such m.a,i»nmems must be handled . Z2, 
measures are to contribute to reliable and practical full i^erence assessments 

Constant spatial and temporal misalignments are commonly encountered in full 
20 ^erenoe test situaBons, and can be <»unte«d by -one or alignment appi, ^t ,h! 
whote re^srence or degraded sequence. Examples of pdor a„ documents whic dea 
wrth such one off alignments are US6483538, US62594r7, US5894324 US6296083 
and USaa7187g. Add«ona,^, fleld^^ed spaBa. or temporaljitter, whe. misalgnm 

25 bis 7 ^ '^'^ '"""'^ •^"'"'""^ ^'^^ - ^ r2 

25 bas.. However, more complex, but equally imperceptible, mls^ignments may also occur 
w.h,n a f,eld or ftame, where d»,e„n. regions o, a video field or frame might L l^Z 

blocKs can occur through video p^cessing and need to be taken into account of if a 

30 °T""' ' '° ^^'^ "sl i 

30 place of human subjective testing results. 

SUMMARY OF TH E |NVFHIT| r»|a 

m..H H ™H T'"' *^ '"""^ by providing a 

method and system for automated video quality assessment which reduces t^e ltrs! 
35 effects of sub-fleldffi,me mbaiignments between «.e -ehrenc, and Z ^eq eZ 
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tot! Zy^T^- T " -l-«9nmenu down ,o a sub-«e,*,ra.a ,evd 

to be handled by IndivKlually matching sub-field/frame elements of a test video fleld*ale 
w,m su^«eld«^„e elements a »fe.noe video «eld*an,e. The us tf a n^^l 
elemen^s^ that Is slgnffloana, smaller than .he video fleld^ame si» enabtes ^JZ 
5 sub-fieldfframe misalignments to be effectively tracked. »les transient 

in view of the above, from a first aspect the present invention provides a video 
quality assessment method, comprislna the steps of 

sub-fie,dr°"T °' - «e«frame with cor^sponding 

'"l>flel*frame elements of at least one reference video field/frame; and 

value in dependence on the matched sub-fiel*frame 
elements of the test and reference video fleld8*ames. ""'"idmame 

The first aspect of the invention provides the advantages described above of 
a lowing sr,b-fleld*ame misalignments and other arte,»cU .o be accounted for such L 

rtt^rT. ""''"'^ ^ " -'-"9™-^ ^"^ othe 

arte^* Which are substantially perceptually unnoticeable ,o a human viewer to be 

sub fleld^ ' '^7^ 0^ --^tehing step preferably further comprises, for a 

20 fledge etement Wi^in M1 preceding and/or M2 succeeding reference video 
ZT"""^- " "'responding reference video fleldm^me to me ,es. video 

f^d^rame. This allows for matches to be made benveen sub^lements no. oniy 

fields*ames, thus allowing fortemporal artefacts such as block freezing ,o be Ln into 
25 account. Preferably *e temporal sea^ llmKs M1 and M2 are predeC airhoug 
venations of the lm,entlon they may be adaptivoly defined. 

sear.h,„*^!," '^'^ embodiment the searching step preferably further comprises 

LTspo 7 " ' ""'^ -^""^ *e 

30 eTemr ates T^r""" "^"^^ » =ub-fio,d.rame 

sean^h withio each fieldfframe then processing ,«qui,»ments are reduced and 

•nsignmcant to a human viewer will be matched. Preferably the spatial extent of the 
35 '"^'■^^ — - - « -y be alZ: 
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In embodiments of the invention the matching step may preferably further 
oompnse oaloulaOng one or more matching statlsfio values and/or matching vectors; and 
Wherein the generating step generates the video quality parameter in further dependence 
on the calculated matching statistic values and/or matching vectors. The use o, matching 
5 stat,s„cs from the matching step has been found to substantially improve fte overa' 
Video quality value which Is obtained, making it more similar to video quality values 
obtained from human subjective testing. The use of matching vectors has similarly been 
found to improve the overall video quality value. 

in embodiments of the invention the generating step preferably further 
10 comprises: - 

calculating a plurality of video characteristic values respectively relating to 
Characteristics of the test and/or reference video fields/frames in dependence on the 
matched sub-field/frame elements of the test and reference video fields/frames- and 

integrating at least the calculated video characteristic values together to give the 
1 5 video quality value. 

By calculating video characteristic values in dependence on the matched sub- 
etements and Integrating such values together a video quality value which takes account 
Of perceptually insignificant video artefacts may be obtained. Preferably the integrating 
step further Includes Integrating the matching statistic value(s) with the calculated video 
20 characteristic values to give the video quality value. Moreover, preferably the video 
characteristic values are respectively any two or more of the following values- one or 
more spatial frequency values; one or more texture values; at least one edge value- at 
least one luminance signal to noise ratio value; and/or one or more chrominance signal to 
noise ratio values. 

25 In a prefen-ed embodiment of the invention, the calculation of the edge value 

preferably comprises, for a test field/frame: 

counting a number of edges in each sub-fleld/frame element of the test 
Tield/frame; 

30 r.f TT^ ' °' sub-field/frame element of the at least one 

30 reference feld/frame matched to the sub-fleld/frame elements of the test field/frame- and 
detemilning an edge value for the test field/frame in dependence on the 
respective counts. 

"~ """""er of edges within sul«lements of the flelds/fremes then 
.mpercepfble differences in edge position can be accounted for, resulting in an edge 
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.^aluafon value wh,ch can be used to derive an Improved video quality value which mo» 
Cosely resembles such a value as derived by human subjec«ve testing 

Preferably, vMn ,he preferred embodiment the mtearatinn «,»n r 

ca,»,^on ,n advani. and ca^^ „ ^^t " ' '"""^ 
.d..3..eri.cval'^ 

a «,ue b,a,ned ,™m a human subjective test. Pre^biy the weighting factors "Tdt 

10 the r ht °" «e ds/frames s h 

10 ma. fo each type of video fleldm^e weight!,^ values speciflc to that type may be us^ 

drrrrr*^"^ ^ '° - ----c 

,ual^ a:rsmn:rn: " ~ ^ - -eo 

15 "'^'*'"8™"«'^^"'«'=Mn9^"Meldfframe elements of a test Video field/frame 

w.h co^spondlng sub.eld.rame events of a. leas, one reference .lotrm: 

.epende::~L^r:.rr:v^r^^^^ 

20 fieldsfframes. '^ferance video 

The sys.em of tt,e second aspeC of presen. InvenBon pmvldes substantlaliv th. 
same advantages as previously described in ,«spec of «,e ^"^stanfaliy the 

Moreover, subsUntiai^ s^iiar fu*er features ard advanirmey al je o^^^^^^^^^ 

30 examplea^ignel being can,edoverane.wcd<suchas.he,nLr!;:, '"^ 

.adab,er:::L:-::rr.Tr^^^^^ 

prog^ms according to the third aspect. The JpJ^r^^ 21 . 
any magnetic, optical, magneto^Bcal, solid-state " 
35 being read by a computer " °«her storage medium capable of 
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BRIEF DESCRIPTION OF THE DRAWINinR 

Further features and advantages of the present Invention will become apparent 
from the following description of embodiments thereof, presented by way of example 
only, and by reference to the accompanying drawings, wherein like reference numerals 
refer to like parts, and wherein:- 

Figure 1 is an overall block diagram of the system of the present Invention; 

Figure 2 is a system block diagram showing the individual parts of the 
embodiment of the present invention; 

Figure 3 is a flow diagram illustrating the steps perfonned to crop and offset the 
input video signals in the embodiment of the present invention; 

Figure 4 Is a diagram illustrating the effects of the matching of sub elements in 
the embodiment of the present invention; 

Figure 5 is a flow diagram Illustrating the steps perfonned in order to match the 
sub-field/frame elements In the embodiment of the present invention; 

Figure 6 is a block diagram of the spatial frequency analyser used In the 
embodiment of the present invention; 

Figure 7 Is a diagram illustrating the construction of a pyramid an-ay In the 
embodiment of the Invention; 

Figure 8 is a flow diagram illustrating the steps perfonned In the constaictlon of 
the pyramid an-ay of Figure 7 in the embodiment; 

Figure 9 is a flow diagram illustrating the steps perfonned in perfomilng the 
spatial analysis on the pyramid an-ay within the embodiment of the invention; 

Figure 10 is a block diagram of the edge analyser used within the embodiment; 

Figure 11 is a flow diagram Illustrating the steps perfonned by the edge analyser 
in.the embodiment of the present invention; 

Figure 12 is a flow diagram illustrating the steps perfonned by the texture 
analyser of the embodiment of the present Invention; 

Figure 13 is a flow diagram illustrating the steps performed by the integrator of 
the embodiment of the present invention; 

Figure 14 is a diagram illustrating a second, software-based embodiment of the 
present invention; 

Figure 15 is a graph illustrating calculated PSNR for no spatial offset (3 Mb/s 
MPEG2 encoding); 
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PPSCfjlPTION OF THE PPFCEPpp.. . ;MBr.niMPKrr« 

Embodiments of the present invention will now be described 

, .K ^ ^ ^'^^ "'"^ ^^sram of the general arrangement 

15 Of *e embodimems Of the inven«on. Within Figure 1 a reference s^uence Zprrs^ng 
referenoe sequence flelds«„mes Input to a detector module 2 Simiiady a s 
sequeno. of ^deo «eids«,«nes 8 (Interohangeabiy referred to herein as either the ^ 
sequence orthe deg^ded sequence, Is also input In to the detector moduie 2 "^I ^ 
sequence « chained by Inputting the reference sequence to a system to be tested (suc^ 
20 as a v,deo recoiling device, a breadcas, system, or a video codec, fcr examp , and 
then ^King the output of the system under test as the test sequence Thr^ictcr 

and generetes video charecterlstic values which are then output to an 
values t<^ether to g,ve a predicted video quality value 1 0, which is output therefrom 
.nvemlon Here Vllr^' " ™" " *^ 

module 32, wh,oh is arranged to receive as Input the degreded ,ndeo sequence 6 and the 
reference v,deo sequence 8. Add«ionaily, the crop and offset module 32 receives .wo 
30 Predetem,inedo«Se.v.,ues,o«setXando«se.yaslnput. The crop and offs "let 
acts to «,„ver. the input sequences to a de^nterlaced fon„at . required. 2 ,^ 0 crop 

"eriroo^rr*'""'""'"*"^''^"'™^'"^ -nertcb:::::^ 

men output from the crop and offset module 32 to a sequence ma,^^ ^ , 
35 Addi«onaily,assho„nihngure2.theoroppeddegreded.dlse:u:nraL^^^^^^^^ 
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from the crop and offset module 32 directly to the degraded sequence inputs of a spatial 
frequency analyser 22. a luminance and chrominance peak signal to noise ratio analyser 
24. an edge detector analyser 26, and a texture analyser 28. The purposes and 
operations of the analysers 22 to 28 will be described later. 
5 Returning to the sequence matcher module 30, the sequence matcher module 

30 receives the cropped degraded video sequence 6 and reference video sequence 8 
and acts to match sub-elements of the degraded video fields/frames with matching sub- 
elements of the reference fields/frames, In order to generate for each test video 
field/frame a matched reference field/frame comprising the sub-elements of the reference 

10 video fields/frames which match to the sub-elements of the test video field/frame under 
test. The matched reference video field/frame 34 is then output from the sequence 
matcher module 30 to a reference input of each of the spatial frequency analyser 22. the 
luminance and chrominance peak signal to noise ratio analyser 24. and the edge detector 
analyser 26. Additionally. In generating the matched reference fields/frames 34, the 

1 5 sequence matcher module 30 produces matching statistics which can be used to produce 
matching statistical values QCPerCent, YPerCent and TPerCenf) which are also output 
therefrom. The matcher module 30 also produces a matched peak signal to noise ration 
value (MPSNR) which is also output therefrom. 

Returning now to a brief consideration of each of the four analyser modules 22 to 

20 28, the spatial frequency analyser 22 acts to analyse the input test video fields/frame and 
matched reference video fields/frames and generates pyramid SNR values PySNR(a. b) 
from a pyramid analysis of the input matched reference fields/frame and the test 
field/frame. Additionally, the luminance and chrominance PSNR analyser 24 compares 
the input matched reference field/frame and the Input test field/frame to generate 

25 luminance and chrominance PSNR values which are then output. Similarly, the edge 
detector analyser 26 analyses the input matched reference fleldAframe and the input test 
field/frame and outputs a single edge detector value EDif. Finally, the texture analyser 26 
analyses the test field/frame to calculate a parameter TextureDeg indicative of the texture 
within the frame. Please note that although not shown on Figure 2 the texture analyser 

30 28 can also receive any of the original reference fields/frames 8, or the matched 
reference fields/frames 34. and generate a parameter TextureRef or TextumMr^f 
respectively relating to these. In any event, the operations of each of these spatial 
frequency analyser 22, the luminance and chrominance peak signal to noise ratio 
analyser 24, the edge detector analyser 26, and the texture analyser 28 will be described 

35 in more detail later. 
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Refemna back to Rgure 1, it will be seen that the output from the respective 

matcher module 30. a„ each input to the Integration module 4, which acts to InLrate 

*evaluestogethertopr«.uce,h.p,edic.edvideo,ua,ityva,ue10. The operati nX 
5 rntegrator 4 will also be described In detail later. 

Refening back to Figure 2, the operations of each cf the Individual modules and 
analysers shown therein will now be described with reference to Figures 3 to 13 

Dealing fl,^ with the crap and offeet module 32, Figure 3 is a flow diagram 

1 u tieldsfframes and reference video fields/frames. 

The general procedure petfomied by the crop and offset module 32 Is to first 
com-ert ,nput sequences from interlaced fom,at to a block deinterlaced fom.at and then 
.0 crap w«h Offset the deg^ded Input sequence and crop w«hout offset the . ere « 
input sequence. Rgu™ 3 lllusfrates an example rautlne which accomplishes these talks 

separately, to produce respecUve capped test and reference sequences. The operatlor^ 
Of the routine is as follows. 

* 3.2 a FOR pracesslng k,op Is started, which causes the crap and 

20 reference sequences). Then, at step 3.4 a frirther, nested, FOR pracesslng loop is 
commenced, which causes the crap and offset module to perfom, the pracees for each of 
the component parts (Y, U, and V, for example) of each fleld*ame 

Wrthin the nested FOR loops, the first action to be periomied Is that of step 3 6 
-^erern the present component part of the input flek«ran,e is first converted ' ii 

YUV422 rnteriaced fbm,at to a block YtW444 delnteriaced format so that each 
. su™ ^ . represented by a,«ys RefV, RefU, and RefV. in accordance 1 1 
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(3-1) 

^ef[yix,y) x = O..X-l, y=>0..y-l 

(3-2) 



10 



(3-3) 

Where X is the number of horizontal pixels within a field and Y the number of vertical 
. pixels. For a YUV422 input, each U and V value must be repeated to give the full 
5 resolution arrays InUField and InVField . 

Next, at step 3.8 an array to hold the cropped and offset pixel values for the 
present component (Y. U, or V) is initialised. The array will be one of RefYfield. RefUfield 
Remeld Degyneld. DegUmid. or DegVfield. depending upon which of the reference o^ 
test sequences is presently being processed, and which of the component parts of the 
10 present frame of the sequence is being processed. A full set of Yfield. Ufield. and Vnold 
an-ays are generated for each input field. 

Following step 3.8. at step 3.10 the region of each field/frame component to be 
copied is calculated, taking into account any offsets to be applied, if any. This routine 
crops With offset the degraded input sequence and crops without offset the reference 
15 input sequence. The offset parameter OffsetX and OffsetY are deten^ined externally and 
define the number of pixels horizontal and vertical that the degraded sequence is offset 
from the reference. The pictur, origin is defined as being in the top left hand comer of 
the image, with a -fve horizontal increment moving right and a +ve vertical increment 
moving down the picture. A value of XOffset=2 indicates that the degraded fields are 
Offset to the right by 2 pixels and a value of YOffset=2 indicates an offset down of 2 
pixels The offset values to be applied can be found by an image alignment technique or 
the like. Image alignment techniques are known in the art. and are described in some of 
the pnor art documents referenced in the introduction. 

For an Input field with YUV values stored in YUV444 format in arrays InYField 
> InUField. and /nVF/eWthe cropped and offset output is calculated according to (3-4) to(3- 

XStart =-QffiietX 

(3-4) 

if {XStart < C^jthen XStart = C, 

(3-5) 

XEM = X -I- OffsetX 

(3-6) 
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if(.XEnd>X-C^-\)then XEnd=X-C^-\ 

(3-7) 

YStart = -QffsetY 

(3-8) 

if(XStart<C,)then YSfart^C, 

(3-9) 

5 

rEiui = y-l-OjB&etY 

(3-10) 

ifiYEnd>Y-C,-l)then YBnd = Y-C,-\ 

(3-11) 

X and Y give the horizontal and vertical field dimensions respectively and Cx and Cy the 
1 0 number of pixels to be cropped from left and right and top and bottom. 

For 625 line video sequences, the following values may be used: 

^ = 720, r = 288, C,=30, C,=10 

(3-12) 

Whereas for for 525 line sequences, the values below may be used:- 

X = 720, Y = 243. C, =30, = 10 

(3-13) 

15 

Xstart, Xend. Ystart and Vend now define the region of each field that will be copied. 
Pixels outside this region are initialised according to (3-14) to (3-17) at step 3.12, where 
YField. UField and VField are X x Y output pixel arrays containing Y, U and V values 
respectively. 

20 At step 3.12 the vertical bars to the left and right of the field are initialised 

according to: 

YField(,x,y)^0 X'=O.XStart-l,XEnd + l..X-l y = 0.y-l 

(3-14) 

UFieId(x,y) = VField(x.y) = m x==O..XStart-l,XEnd+l..X-l jK-O.Jf-l 

(3-15) 
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The horizontal bars at the top and bottom of the field are initialised according to: 

YField{x,y) = Q x = XStart.JCEnd, y = O.J'5/ar/-l, J'£nd + l.j'-l 

UField(x,y) = VField(x,y)^nS x^ XStart..XEr,d y = 0..rStart-l,YEnd+l..r-l 

Finally, at step 3.14 the pixel values are copied according to: 

YField{x,y) = InYField{x + OffsetX,y + OffsetY) x = XStart.JCEnd y = YStart.YEnd 
UField{x,y) = InUField{x + (WselX,y+OffiietY) x ^ XStart.JCEnd y^YStart.YEnd . 
VField{x,y) = JnVField{x-^XQffiet,yArYOffiset) x^ XStart.JCEnd y^YStart.YEnd 



(3-16) 



(3-17) 



(3-18) 



(3-19) 



(3-20) 



15 For the degraded Input, cropping and shifting produces output field arrays DegYField, 
DegUField and DegVField, whilst cropping without shifting for the reference sequence 
produces RefYField. RefUField and RefVfield. These X x Y two dimensional arrays are 
then used as inputs to the matching module 30 and analysers 22 to 28 as. described 
below. 

20 The cropped test video fields/frames and reference video fields/frames are 

output from the crop and offset module 32 to the sequence matching module 30. In order 
to perform the matching process, the sequence matcher module 30 must receive several 
fields/frames of each of the reference and test sequences 30. and buffer these 
fields/frames. Preferably, as many reference and test fields/frames sequences are 

25 buffered as the matching module 30 has buffer space. The operation of the sequence 
matching module 30 is shown in detail in Figure 5. 

Firstly, at step 5.2 the matching module 30 receives the cropped reference and 
test fields/frames sequences and buffers the individual fields/frames, as mentioned 
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abova Then at step 5.4 a processing loop is started for each of the test video 

^.n me process,ng loop, the next step for the present fleld/,ran,e N Is pertonned at 
step 5.6 whe,«n the present fleld/,«me Is split Into by ,y pixel blocKs. where both 

^ ^T""^ ^ ^"^^ I' -n^-^" by .he fleld/fra™ A, In which I 
belongs and the top len coordinate (x, y, of that block. Then, the next step, a, step 6 8 
s 0 start a second processing loop to process each pixel block (A/, Px, where N 

I Z IZ ' '"'^ ^ '"^ °' '"^ ^ 

10 Having commenced the processing of each pixel block within the present 

fleldfl^me. a. step 5.10 a further processing loop is commenced wherein each neZTe 

fields«.mes pnor to the present frame N under test ,o be searched, as well as M2 
succeeding fields/frames. Next at atao-i 17 = wen as m.^ 

1S , ^ \^ . "•^'^^•^2 a further processing loop is commenced to 

15 ^ar* each honzomal pixel position x between the search boundaries P. - u to P. . 
Oc, and then further at step 5.14 yet another pracessing loop Is commenced to process 
each ver^ca, pos»on y be^«en the boundanes ^ - to * Ly. The effect of each o 
the pn,ce^,ng loops commenced at step 5.10. step 5.12. and step 5.14 Is to cause an 

20 ,0 be search^l ,n each framefr«n Ml preceding frames to M2 succeeding ftames to the 
reference flekl^rame which temporally conesponds to «,e present ,le,d*ame under test 
This ,s Shown ,n Figure 4. wherein the pixel block B shown in fleld N of the degraded 
sequence w,th bottom left pixel coo„:llnate (x. Is searched for in each of Ja^a! 
indicated within the reference sequence. 

26 •^«^P«'»™»='«<' search fbr the pixel blocks Within the indicated areas of 

me reference sequence is that a matchl,^ error B(n...y) is detemiined for each position 
withm the sean*ed areas at step 5.16. In accondance with the following:- 

bx-l by-l 

x=Px-Lx,..,Px,..,Px+Lx 
y = Py-Ly,..,Py,..,Py+Ly 

30 (5-1) 

The matching error that is determined at step 5.16 is stored for the present 
values Of n. x, and .for later use. Next, at step 5.1B the value y is incremented uZ it 
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has reached rts maximum, and process!,^ returns to step 5.14, wherein the matching 
error « detem„n9d for the next value of y. I, y has reached its maximum value then 
processing proceeds to step 5.20, wherein x Is Incremented unless i, has mached Hs 
maximum value, and processing proceeds back to step 5.12. If x has reached its 
5 maximum value then processing proceeds to step 5.22. wherein the counter n Is 
incremented unless it has reached Its maximum value as defined in step 5 10 and 
Processing proceeds back to step 5.10 , appropriate. If „ has reached its maximum 
value, then processing proceeds to step 5.24. The effect of the nested FOR loops of step 
5.ia 6.12 and 5.14 Is that a matching error E(n. x, y) is detemiined for each possible 
10 position within the search regrons shown in Rgure 4. 

Having obtained the matching em,r values E(n. x, y) for each possible position 
the next step at step 5.24 is to find those values of n. x, and y wherein the error value is a 
minimum. This Is performed by sorting the stored matching error values to detemilne the 
minimum enx>r values, and then detemiinlng the values of n, x, and y which led to that 
15 minimum value I.e. 

Lx,.., Px,.., Px + Lx 
y=Py- Ly,.., Py,.., Py + ly 

(5-2) 

once the minimum matching error has been determined in this way. the matched 

20 tH r '""^ y-^' Which is 

20 then the matching block within the reference sequence to the present block being 
processed within the test sequence. 

Having found the present matching block, at step 5.26 the matched block 

riTL? ^""^ ' '^"'"'■""^^ ^^^^y oon-esponding to the present 

test fieldAframe. and In particular is copied into the block within the matched references 
25 array which has the identical position to block B^N, P., Py). That is. the matched block 
from the reference an^y. In-espective of from where in the search region it was taken is 
copied into the corresponding block position within the matched reference field/frame 
being constructed as the present block under test has within the present test field/frame 
in accordance with the following:- 

30 MrefiN.Px^j,Py^^-)^r^^„^^^^ ^j^^^ J^O..J>^-l.k^O..J>, -, 

Where 6 is the dimensions of the block (ibx and /by). ^^"^^ 
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Next, at step 5.28 processing is caused to move on to tlie next pixel blocl< in the 
present test field/frame, if appropriate, wlierein the next pixel block is processed as 
described above. The processing of every pixel blocl< within the present test field/frame 
as described above results in a matching reference block being found for each of the test 
5 blocks, and the copying of the matching reference block into the matched reference array 
causes a matched reference field/frame to be produced for the present test field/frame. 
Thus, once every pixel block has been processed accordingly (as detemiined by step 
5.28). at step 5.30 the newly constructed matched reference field/frame can be output to 
the respective analysers 22 to 28 as appropriate for use thereby. Moreover, by repeating 

10 this process for every field/frame in the test sequence, a sequence of matched 
fields/frames can be constmcted for use by the analysers. 

The matching process of first searching for the best match for a degraded block 
followed by the copying of the resulting block into the matched reference array is 
repeated for the whole of the desired analysis region. This analysis region is defined by 

1 5 block centre points Px() and PyQ according to: 

Px(Jh)'=KX\-\-KX2*h A = 0..er-1 

(5-4) 

and 

Py(y) = Kn-¥KY2''v v = 0..j2>'-l 

(5-5) 

Where Qx and Qy define the number of horizontal and vertical analysis blocks, KX1 and 
20 KY1 define the initial offsets of the analysis blocks from the top and left respectively and 
KX2 and KY2 define the analysis block shifts. 

The matching analysis of the N'th field therefore produces a matched reference 
sequence described by 

BlockM Re fiN, Px(,h), Py(v» h = O.-Qx - 1, v = O..Qy-\ 

(5-6) 

25 and a set of best match error values 



A set of offset anrays MatT, MatX and MatY can be defined such that : 



(5-7) 
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h = O..Qx-l, v = 0..2y-l 



Table 1 



The matching parameters for 625 and 525 broadcast sequences are given 



(5-8) 



in 



Parameter 


625 


526 


Qx 


87 


87 


Qy 


33 


28 


KX1.KY1 


12 


12 


KX2.KY2 


8 


8 


Lx.Ly 


4 


4 


bx,by 


9 


9 


M1 


4 


4 


M2 


5 


5 


M 


2871 


2436 



Table 1 Search parameters for matching procedure. 

The analysis region defined by (5-4) and (5-5) does not cover the complete field 
10 size. MRef must therefore be Initialised according to (5-9) so that it may be used 
elsewhere unrestricted. 

MRe>f(x,y) = 0 jr=:O.JSr-l, y = 0.j'-l 



15 



20 



(5-9) 

Returning to the flow diagram, however, having output the newly constructed 
.ndividual field/frame to the analysers at step 5.30. the matching module next proceeds to 
calculate a number of matching statistic values and other values from the matching 
process which has just been perfonned. and the calculation of these matching statistic 
values and the other values is described next. 

The primary function of the matching module 30 is to produce a matched 
reference sequence for use within the detection module of the video quality assessment 
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H^r: V H " ""^ '^'^ '^^'^ above. 

However, rt has also been found that some neasu^s derived wHhln the matching 

P^cess can also be of use In p^duclng the final video qua,,,, value. One suoh measure 
that ,s related ,o the horizontall matching statistics. Is described belov„. 

At step 5.32. horizontal matching statistics from the matching process are 
calculated for use in the Integration process. The best match fOr each analysis blocK 

ItTT " = •''^'"S-nn histX fo; 

each field according to: 



(5-10) 

Where array histX is initialised to zero for each field. The histogram Is then used to 
detennine the measure fXPerCent acconiing to: 

/WerCen(.lCKI»Mir(4i,Or(())/g«,iJ-t,) ( = 0 2i. 

15 

bloc., "^"^ "^'"^ W °f 

blocks ha. comnbute ,c the peak of the matching histogram. Tl* KPerCen. measum is 
output from the matching module 30 to the integraflon module 4, for use later as will be 

described. 

m tc ,ng module 30. Here, the minimum enor, BQ. for each matched block is used to 
calculate a matched signal to noise ratio according to: 

25 (5-12) 
S AAT. i'^CA). Py(y))) = 0 = 10 Iog,.(255^) 

(5-13). 



i>«0 
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7T>e thus deten^ined matched PSNR for the pr«en. fleldm^me Is then output to the 
integratron module 4, for later use thereby to be described U« to the 

sto^df Tf'"' ^^'^ 8«"-^ed and 



10 



(5-14) 

sv~jr(«.v).4&ur(*,»)-/i(4) ».o.^-^ v.o.a,.-i 
«i™i'(*.v).Mal'(*.v)-iM(.) A-o.^.-!. ,.o..e,-i 

(5-16) 



15 



In summaor, therefore, from the above It will be apparent that for each 
field*ame ,n the test video sequence, a matched reference fleld*ame can be found by 
matching sul«lements of the field*ame with sub^lements of the fields/frames of the 
reference video sequence, and furfhsmiore that matching statistic values and other 
20 measures de^ed fi™, the matching process can also be output, and which are used in 
the integrated module 4, as will be described later, 

of erther tt,e luminance value (Y) of either of the chramlnance (U or V) values. However 
It Should be noted that the same precess Is applied separately to each of the Y, U, and V 
25 p«el values to generate separate one dimensional matched sequences for each of these 
component of the colour space. Thus, effectively, the process of Figure 5 is repeated 
separately for each otthe Y pbcei values, U pixel values, and V pixel values 
nro.. '""^•'l^nts. Of course, a single three dimensional matching 

30 oir, ™' "'^'^^ "< colour 

30 ^rameters IS currently the preferred and tested .echnK,ue for all of the matching process 

Of course the matching technique as des»1bed may also be applied to other colour 
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spaces such as YCbCr or Rrr xa/wk «r,i., 

skilled m the art ' """" "^odlflcatlons appa,,nt to the person 

rr;:: ::rr: rerr ir 

equenoes to be p^oessed. For example, the Inventors have found l^r LTnT 
76 .ntertaced 60 fleMs per seoond, a™. 526 (720 by 486 interlaced 60 fle ds Lrsel„d 
16 h^adcas. se,uenoee then the spatial parameter U and should be 4 Z2 
the temporal parameters Ml and M2 should be set to 4 =nH r- 
Mo.ov.r,a n^a.cHI,^blooks,zeo,«b,ap.e,sl:L:pr:t:::; 

Returning to Figure 2, It will be seen that the matched reference sequence 34 
output from the sequence matcher module 30 is input to each of the .1!!? 

°— -'-^ 

fh» . """^ "'^ '° 22, the Internal conflguration of 

1-^"""°' '""'^'^ ' '-nsfon. one a?o r 

.222 wh,ch « a^nged to receive as an Input *,e test video fields/frames Additn 

30 xrrra— ^^^^^^ 

pyramid snr measures will be desortbed nex,.^ lence TZ^Z r^"' 

36 perfon.erreIeTo;r:^rr; ^ ^ "^-^ *e steps 

r Of the pyramKj transfom> generators 222 or 224 In producing 
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r^pecve pyramid anays, The^ore, firstly a, step 8.2 ,hs pyramid .ransfom, generator 
^-es an inpu. fie,d*,me m=m me ^.pecive sequence O.e. .es. sequence or ma" d 
mference sequence output from the matching module 30,. Then, at step 8.4 a count^ 
slsge ,s ,n,t*ed to ^ and a pnx=essing loop commenced in order to generl "he 

^ge. *™ step pmcedure, whe^ln for each stage 0 to 2 horizontal analysis is performed 
followed by versoa. analysis. T.e steps involved In one particular stage of horiTont and 
vertical anab-sis are descnbed With aspect to steps 8.6 to 8.20 next ^ 
Once within the processing loop commenced at step 8.4, for a particular staae of 

10 pyramid processing the firet step perfomied at steo 8 e fh=, ,k ^ ^'^^^ °' 

L. . K^iiuiiiieu ai siep b.6 IS that the present field/fram^^ 

being processed is copied into a temp array, as follows:- 

PTemp(x,y) = P(x.y) x^O..X-l. 

Then, at step 8.8 the horizontal analysis limits are calculated as a function ofthe 
present value of the stage parameter as follows:- 

23c = Jir/2^'*«'*»> 



(8-2) 



20 ""'^ P^*™^ '^^'"'la'^'' ""-Its, SUCht!! 

t3^ r -""^ °' »' '-"-'V ai^y a. ured 

to update the pyramid an-ay according to: 

(8-4) 

Z:^^:^^" "'^^-"^^ — With the results of^ 

Vertical analysis for the present stage of processing Is then perfomied 
commencng at step 8.14 wherein the input field/frame is again'copiedL IhtZp 
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array. However, at this point it should be noted that the values within the input field/frame 
were overwritten at step 8.12 with the results of the horizontal analysis, and hence it will 
be seen that the input to the present stage of vertical analysis is the output from the 
immediately preceding present stage of horizontal analysis. 

Next, at step 8.16 the vertical analysis limits are calculated as a function of the 
stage value, as follows 

(8-6) 
(8-7) 

Following which vertical analysis Is perfonned within the calculated limits 
according to the following, at step 8.18 so that averages and differences of vertical pairs 
of elements of the temporary an-ay are used to update the pyramid array according to: 

(8-8) 

n*,y^Ty)^PTemp{x,2y)-PTemp{x,2y^\) x = Q.Tx-\ y^Q.Ty-\ 

(8-9) 

At Step 8.20 the input field/frame is ovenwritten with the results of the vertical 
analysis perfonned at step 8.18 such that the values within the input field/frame array 
con-espond to the results of the first stage of the spatial analysis. At step 8.22 an 
evaluation is perfomied to detemiine whether each of the stages of the spatial analysis 
to generate the pyramid array have been perfomied. and if not processing returns back to 
step 8.4. wherein the stage value is Incremented, and the steps of 8.6 to 8.20 repeated 
once again. It should be noted that for each step of horizontal and vertical analysis at 
each stage, the values within the Input field/frame array are overwritten with the 
calculated vertical and horizontal limits, such that as processing proceeds step by step 
through each stage, the values held within the input field/frame array are converted into a 
pyramid structure each of four quadrants at each level. Thus, by the time each of the 
stages 0 to 2 has been completed, such that the evaluation at step 8.22 causes the 
processing loop to end. a pyramid an-ay has been constructed which can be output at 
step 8.24. 
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= sp« Ho four ,uad,Bn.s Q(.,a,e, o .0 3, whe^ Q,o T ^ 
.0 .he ava..e o, Hoc. . 4 p... j l^lT ~ ^^^^ 

coTOsponding to the horizontal dlffe,«nee of blocks of 4T . 
Q(0, 2) contains values corresoondinn 1 *^ fl»l*(«nie, 

-3>--.scor:nrtrr~=^^ 
■ --"»::rror2:^^^^^ 

processing, «,e re,u«s of which a« shown In B^reZ H f ^ °™ 

quadrant 0<0, 0) has been ove^ttten by Z^ oa oti s, T h *^ 
o' 4 by 4 p^e, blocks, but whe^n each [JZZ o lT'^ '^'^ ^"^'^ 

15 the average, horizontal d«e«m« ve^cal d„ ' '^'^"^ '° 
P^v^us^descnbedlnrespectofresTetuZ ""^^^ « 
The output of the stage 1 analysis as shown In Flour. 7,m . 

stage 2 analysis In the tt,irt Oration of the fZ Ioo^Tp '"^^ '° 
shown in Figure 7(c) where It will h. T .u . . '""'^ ^' '° 8'™ "'"1^ 

20 by the quadrants Q 2 ^^3, 2re 1^ °' ~- 

w^^, u to 3;, where each of the quadrants on n , . 

average of blocks, horizontal diffemnce of blc^s 1 r 

previously. After the th>ee stao., „f , respectively as described 

Figure 7(c, has a to^i bio* r T "^""'^ » 

3.a.e 0 C2 by 2 ^^^.^Z^:^:':^'^ ^r,^^: °<°. ^ 3, fro. the 

25 — ,andfour,ua<*an.sO(^0to3,,.:\e«^^^^^^^^ 

be noted that the procedure of R™„» a . . ^""'S'^'*- « ^l^ould 

r IS pe*m,ed by eat " I pZi^T T ^'"^^ '^"^ "^ure 

respective pyramid arrllranrT ^ T" '° 

30 Wrth reference to Hgure 9, firstly at step 9 3 

receives the reference and degraded ]^1^ZZ 

generators 224 and 222 raspe«^«v. Ne« at ^epTTa pr^sJ 

Which processes each vaiue of the counter vaiue^ ^ ! 

second, nested, precessing loop which precesses a cou^rr ! ^ ' ' 

35 ™,ueso,t.o3lscom.encedats.epa.e. W«hin .hL~ r::^;:^; 
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9.8 a squared e^r measure value E(s,age. Is calculated beW,een the 

reference and pyramid anays, aocorting to: 

EM^iUXY^) t^i: (Pre/(..j.)-Pdeg(..3,))^ . = 0 2 ^-13 

(9-1) 

5 Where x2, y1 and y2 define the horizontal and vertioa, limits of the quadrants within 
the pyramid an-ays and are calculated according to: 



10 (9-2) 

^l(..2)=0 .2(..2) = ^/2(-> ;.l(..2) = r/2'-) y2is.2) = 2* yKs^l) 



25 



.lW) = ^/2-> .2(.^) = 2-x,(..3) ;,l(.,3) = 7/2<-> ^2(..3) = 2 ^IW) 



(9-3) 



(9-4) 



Each calculated en-or measure Ersfage. quadrant) is then stored at step 9 10 
0 ow.ng Which at steps 9.12 and 9.14 the values of the quadrant and stage counter, are 
updated as appropriate to the processing loops. The operation of the processing loops of 
step 9.4 to 9.14 and step 9.6 to step 9.12 Is to calculate an error measure value for each 
value of the counter stage and the counter quadrant. 

Having calculated the squared error measure values, at step 9.16 a further 
processing loop to process all the available values of the counter stage from 0 to 2 is 
commenced, following which at step 9.18 a nested processing loop to process the values 
o the ^-c/^"^ counter 1 to 3 Is commenced. Within these nested processing loops at 
step 9.20 a PSNR measure PySNR^stage. quadrant) Is calculated according to:- 

//(^>0.0) i>SAW(.,^) = io.OMog,(255V£(..^)) else 5;N« = 10.0*log,(255^*^^) 

Which is then stored at step 9,22. At steps 9.24 and subsequent step 9.26 the valu^!! 
he coun ers sfa^e and quadrant are Incremented as appropriate to the processing loops 
such that the effect of the nested processing loops is to calculate and store the PSNR 
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measure for each value of ^a,a and each value of ,ua,r.m. Given «,a. *e parameter 
.tese can take values of 0 ,o 2, arK. the pa-ameter ,u^r^t may take values of 1 to 3 It 
be seen «,a, a total of 9 PSNR measu^ are generated by the pyramk. SNR 
calculator 226. all of Which may be output to the integration stage 4 
5 The operation of the edge analyser 26 will now be described with mspect to 

Figures 10 and 11. h«w lo 

Figure 10 Illustrates the Internal configuration of the edge analyser 26. More 
part,cularly, the edge analyser 26 comprises a first edge detector 262 arranged to receive 

10 2^ T ^ '° ^ ^ --"^ ~ 

264 ananged to receive the matched reference video tiekis*ames output from the 

matching module 30, and to detect edges therein. Both the edge detector 262 and 264 

prel^ably operate in accoKlance with known, edge detection algorHhnis and produce 

edge maps ,n a manner already known in the art For example, examples of known edge 

15 edge detectors. Source code in the C programming lar^uage for a Canny edge detector 
2 available for free download via ftp l^ore the priomy date from 
fl p7/ftqmenl.Ci«e,isfr^u/p„IVgdn. r^n.,nHon/»r.nr^ ^,,,^3 
code ,n c for a Rothweli edge detector was available from 
Sp7/flament.cseensf^,i»lih>Fdne Comn.ri^n/.„. cndor,o.hw,ii „- 

The respective edge maps praduced by each of the edge detectors 262 end 264 
are input to a block matching means 266 which acts to compare the respective edge 
maps In a manner to be described, and to praduce an output parameter HD/f 
representative of the comparison. The operation of the edge analyser 26 Is shown Iri 
more detail in Figure 11. 

262 and'T "'"^ ^ '''^ "~ '^^^ ^ 

262 and 264 calculate respective reference and degraded edge maps. As mentioned 

above, the edge detection algodthm used by the edge detector 262 and 264 Is 

preferably one which Is known in the art, such as a Canny edge detector. The edge 

30 "'^"'^ -'P- «° *e block 

30 matching means 266, wherein at step 11.4 each of the reference and degi^ded edge 
maps are split Into A, by « blocks. Next, the block matching means 266 acts to Jnt 
each Pixel *ch forms part of an edge within each block In both of the reference and the 

ZTo,T "T' '"^'^ ""^^'^ 

count Of edge p«els for each block In each of the ™ferenoe and degraded edge maps ' 
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TZr T rr"* "^^^ ^'^^ " - -"V d^erence 

Following step 11.8. a, step 11.10 the block matching means 266 puts eacl, 
difference value to the power Q and at step 11.12 the resuming values are summed 
Therefore, after step 1 1.10 there are stiii as many values as there are blocks in one of the 
-Bfemnc. or deg,«,ed edge maps, but after step 11.12 a single result is obtained 

10 oorrespondmg to a sum of the values calculated at step 11.10. At step 11 14 the 
resulting sum value Is then put to the power 1/Q. and at step 11.16 the result oi this 
oalculat,on Is output from the block matching means 266 as the ED,r parameter. As will 
be seen from FIgum 2, the EDff parameter Is output from the edge analyser 26 to the 
rtegr«,on stage 4. Use of the EDif parameter wl«,ln Uie Integration stage will be 

TO aescnbed later. 

It may be useful In some situations to take Into account analysis offsets from the 
flekl^ame edges In the edge differencing steps of. 11.6 to 11.16, in which case the 
processing then becomes as follows. 

20 , . ""^^^ respective .edge maps, the block matching means then 

20 calculates a measure of the number of edge-marked pixels In each analysis block, where 
nX and nY define the number of non-overlapping blocks to be analysed in the horizontal 
and vertical directions and XI and Y1 define analysis offsets from the field edge 



26 



(11-1) 



Ml ton 



+J) x = 0.jtX-l,y = 0.jtY-i 



(11-2) 



30 



The summation limits are detemilned according to: 

n==-(N dtv 2) i2 = {N-V) dtv 2 



(11-3) 
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Jl = ~iM div 2) J2 = iM-l) div 2 



(11-4) 



Where the "div" operator represents an integer division. 

Then, a measure of the differences over the whole field is calculated according 

5 to: 

(11-5) 

For 720x288 pixel fields for 625 broadcast video: 

^' = 4, Xl = 6. nX = 17B, M = 4, n = 10, «y = 69, 2 = 3 

(11-6) 

1 0 Whereas for 720x243 pixel fields for 525 broadcast video: 
N = 4. Xl=6. nX = 17S, M = 4, n = 10, nr = 58, 0=3 

(11-7) 

It should be noted that the above processing represented by equations 11-1 to 
11-7 is substantially identical with that already described in respect of Figure 11. with the 
differences that the analysis offsets from the field/frame edges are taken into account. 
15 The parameter Ed/f found by equation 11-5 is output to the integration stage 4 in the 
same manner as previously described. 

The operation of the texture analyser 28 will now be described with respect to 
Figure 12. 

Digital video compression tends to reduce the texture or detail within the an 

20 image by the quantisation of the DCT coefficients used within the coding process. 
Texture analysis can therefore yield important information on such compression, and is 
used within the present embodiment to provide a video characteristic value TextureDeg 
and/or TextureRef. More particularly, the texture parameter values TextureDeg and/or 
TextureRef are measured by recording the number of tuming points in the intensity signal 

25 along horizontal picture lines. This is perfonned as shown in Figure 1 2. 

With reference to Figure 12. firstly at step 12.2 the texture analyser 28 receives 
the present test field/frame to be processed. From Figure 2 it will be recalled that the 

^ texture analyser 28 receives the test video field/frame from the crop and offset module 
32, but does not receive either the matched reference field/frame or the original reference 

30 field/frame. However., in other embodiments the texture analyser 28 may receive either of 
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the matched reference field/frame or the original reference field/frame in which case a 
rexf.;^Ref parameter may be calculated in the identical manner to be described below In 
respect of the TextureDeg parameter. 

Following step 12.2. at step 12.4 a turning point counter sum is initialised to zero 
5 Then, at step 12.6 a processing loop Is commenced for each line within the input video 
field /frame loop within the limits loop = 0 to 1. wherein Y is the number of lines within 
the video field/frame. Within the processing loop, at step 12.18 values last_pos and 
lasLneg are both Initialised to 0. Next, at step 12.10 a second, nested, processing loop 
.s commenced to process each pixel x within each line y. where x takes the value of 0 to 
10 X-2. wherein X is the number of pixels In a line of the input video field/frame 

Within the nested processing loop, at step 12.12 a difference value is calculated 
between the pixel value at position x. and the pixel value at position x -M. Then at step 
12.14 an evaluation Is perfomied to detemilne whether or not the calculated difference 
value is greater than 0. and also as to whether or not the value last^neg is greater than 
15 the value last_pos. If -this logical condition is met then the counter value sum is 
incremented. Following step 12.14. at step 12.16 a second evaluation is performed to 
detemiine whether or not the difference value calculated at step 12.12 is less than 0 and 
as to Whether or not the value lasLneg is less than the value /asU,os. If this is the case 
then the counter value sum is Incremented. It will be noted that the evaluations of step 
20 12.14 and step 12.16 are mutually exclusive, and that it is not possible for the counter 
value sum to be Incremented twice for any single particular pixel. After step 12 16 at 
step 12.18 a further evaluation Is determined as to whether or not the calculated 
difference value is greater than zero. In which case the value lassos is set to be the 
number of the cun^nt pixel x. Alternatively at step 12.20 a second evaluation is 
25 performed which evaluates as to whether or not the calculated difference value Is less 
«ian zero, in which case the counter value last_neg Is set to be the cun-ent pixel number 

Following step 12.20. at step 12.22 an evaluation Is performed to determine 
whether or not ail of the pixels x within the present line have been processed, and If not 
30 then processing proceeds back to step 12.10 wherein the next pixel Is processed 
However if ail of the pixels have been processed then processing proceeds to step 
12.24. wherein an evaluation is made to determine whether or not all of the lines y have 
been processed in the present input frame, and if not then processing proceeds back to 
step 12.6. When processing of the next line is commenced. The results of these nested 
processing loops are that each pixel on each line is processed, and whenever the 
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value Which ,s ,nd,oaSve of the .extu,e turning points wnhin the input fleldfframe 

, '^^'"3 ^^'"^ "«hln *• counter sum, at step 12.26 a texture parameter 
5 ,s celouiated as a ft.nc«on otthe value held in the counter sun,, as follows: 

Texture = * 100/;ey 

(12-1) 

10 28 to th ^^^-^ t-^-^e analyser 

10 28 to the integrator stage 4 at step 12.28. 

^""'^""°"«*=l»«'"9h Figure 2 illustrates the texture analyser28 mcelving only 
Z7T "Th~"^-' P™^""'"^ a rexf..z,e.paran,e. liue 

v^eo fields/frames or the matched reference video flelds*an,es h add«ton. whe^in the 
TaxtureMref parameters in addition. 

an=,„ I^' r'""" °' I»ak Signal to noise ratio 

analyser 24 will now be described. 

20 anah»».?/'°"" ^ '""""'"^^ *^<""'"="'» P»«K ^iQnai to noise ratio 

~ ~ "^'^"^"^ <'^-'«' Video 

«e ds/frames as .nputs. These can then be used in the imensity and colour signals to 

ZLTr^'"^" '° "^^^^ "^'^ ara flelds of 

matched ,«,erance and degraded Intensity and RefU, DegU, RefV and DegV ara flelds of 

chrominance according to YUV standard colour foniiat:- 



(2-1) 

r-i 



(2-2) 



x«0 j^O 
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(2-3) 



Of course, In other embodiments of the invention which do not use the YUV 
colour model, such as RGB. and YCbCr. then of course similar corresponding 
5 measurements may be calculated as will be apparent to those skilled in the art. 

Additionally, the luminance and chrominance peak signal to noise ratio analyser 
24 also uses the sets of matching vectors generated in the matching module 30 to 
generate a segmental PSNR measure SegVPSNR. relating to the V values of the pixels 
When a YUV colour model is used. More particularly, a matched signal to noise ratio is 
10 calculated for the pixel V values by use of the matching vectors defined in (5-14) to (5- 
16). above. For each set of matching vectors, an error measure, VE, is calculated by the 
luminance and chrominance peak signal to noise ratio analyser 24 according to- 



VE{h,v) = (X/bx * by)J] (OegVCN.PxCh) + i,Py(h) + /) - 

/"O 7-0 '''' 

K^mN + SyncTQt, v), Px^h) + SyncXih.v) + i.Pydv) + SyncYih.v) + j)^ 



-fc ^ (2-4) 
15 ind.a segmental PSNR measure is then calculated for the field according to: 

SegVPSm=(^/Qx*Qy)^ S 10.0*log„(255V(KE(A,v) + l)) 

As=0 vbO 



The segmental PSNR measure SegVPSNR is then output to the integration module 4 fo! 
later use to be described. 

Returning to Figure 1. the various outputs from the matching module and 
analysers within detector module 2 are fed to an integration stage 4. wherein the various 
values are integrated together to give a video quality value 10. The operation of the 
integration stage 4 will now be described with respect to Figure 13. 

Generally, the operation of the integration stage is to produce an estimate of the 
25 perceived video quality of the test video sequence by the appropriate weighting of a 
selection of the video characteristic parameter values produced by the analysers 22 to 
28. The particular set of parameter values used and the values of the corresponding 
weighting factors depend upon the particular type of video being tested and are 
determined in advance by prior calibration. The calibrations are perfomied on a large set 
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Of video sequences tt,at have known subjective scores, and preferably have properties 
similar to the degraded sequences to be tested. 

K r ^^"^'^ '^'^ •"^'1' ""-^ '^'9'''^ »e flel*frame 

by field/frame detection parameters, and then combines the tlm^weighted and averaged 
5 values to give a predicted quanty score, being the overall video quality value The 
process to achieve this is set out in Figure 13. 

Firstly, the integration stage 4 receives the parameter values output from the 
vanous detectors and analysers at step 13.2 and stores them. As has been described 
previously, the matching module 30 may output the various PerCent and MPSNR values 
10 the spatial frequency analyser 22 outputs the PySNR values, while the luminance and 
chrominance peak signal to noise ratio analyser 24 outputs PSNR values for each of the 
luminance and chrominance characteristics In the colour model being used as well as 
the SegVPSNR value calculated from the matching vectors. Moreover, the edge 
analyser 26 outputs the ED^ parameter as described previously, whereas the texture 
15 analyser 28 gives the values Textur^Deg at least, but might also output values 
TextureRef an6 TexturoMref, appropriate. Whatever parameters and values have been 
output by each of the earlier stages In respect of a particular test video field/frame the 
integration stage receives the output Information and stores it. 

Next, at step 13.4 the integration stage selects the video type, and as a result 
20 selects a set of integration parameters In dependence on the video type. For example a 
set of integration parameters for 720 by 288 pixel per field 625 broadcast video that has 
been MPEG encoded at between 1 Mbits per second and SMblts per second Is given 
below: 
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K 


Parameter name 


w — 


0 


TextureDeg 


-0.68 


1 


l-'ySNR(3,3) 


-0.57 


2 


bUif 


+58913.294 


3 


fXPerCent 


-0.208 


4 


MPSNR 


[-0.928 


5 


aegVPSNR 


-1.529 


Offset 


+176.486 


mnk 


To ' ■ 
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Table 2 Integration parameters for 625 broadcast video. 



Whereas the weighting values for 525 line video are:- 



K 


Parameter name 


W 


0 


TextureDeg 


+0.043 


1 


PySNR(3,3) 


-2.118 


2 


EDif 


+60865.164 


3 


FXPerCent 


-0.361 


4 


MPSm 


+1.104 


5 


SegVPSNR 


-1.264 


Offset 


+260.773 


mnk 


1.0 


N 


480 



Table 3 Integratlori parameters for 525 broadcast video. 



The precise values of the various weighting factors are determined in advance 
by calibration, as described. Moreover, each set of integration parameters is stored 
within the integration stage 4 in look-up tables or the iike. 

Having selected the video type and set the integration parameters from the 
stored look-up tables, at step 13.6 a processing loop is commenced in order to process 
each integration parameter type /f within the values 0 to K-1, wherein each parameter (K) 
is a particular one of the parameters received from the various analysers or the matching 
module. Within the processing loop, at step 13.8 firstly a time weighted average AvD(/c) 
of the parameter values is calculated according to the following:- 



^v£»(*:) = (l/AO*(j;D(;fc,„)"*)"«»» • 



(13-1) 
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where N is the number of fields. D(k. n) is the n 'th field of the /f-th detection 
parameter, and mnk is a weighting factor. Next, at step 13.10 the time weighted average 
value AvD(k) is multiplied by the appropriate weighting factor w(k), and the product 
stored. The appropriate weighting factor w(k) is read from the appropriate look up table 
5 for the video type stored in the Integration stage 4. 

At step 13.12 an evaluation is performed to determine whether or not all of the 
integration parameters (k) have been processed, and if not the processing loop of step 
13.6 is performed again until all of the parameters have been processed. Once all the 
parameters have been processed then an appropriately, weighted time weighted average 
10 value will be available for each type of parameter k. which are then summed together at 
step 1 3. 1 4 with an offset value as follows:- 

PDMOS = Cng&et + AvD(k) * W(k) 

(13-2) 

where K equals the number of parameters used in the integration and is 6 for the 
examples shown in Tables 2 and 3.. This gives a final video quality value PDMOS, which 
is then output at step 1 3. 1 6. 

The output video quality value PDMOS Is equivalent to a rating produced by 
subjective testing using human observers, in that it has been produced taking into 
account distortions and en-ors in the test video signal which are substantially perceptually 
undetectable by human observers. Therefore, in view of this property the video quality 
value may be put to a number of uses. In particular, It may be used to evaluate the 
quality of an existing video service to ensure that the quality is adequate, or alternatively 
It may be used to test the perfomiance of different video codecs. Additionally, the video 
quality value may be used to evaluate the perfomiance of new video services, such as 
broadband-style video services over the Internet. In this respect, the video quality level 
PDMOS generated by the embodiment of the present invention may be put to any similar 
use as those automated quality assessment values generated by the prior art, with the 
difference that as the value takes into account perceptually insignificant distortions, it is 
much more likely to represent a video quality value generated by human viewers during 
subjective tests then has heretofore been the case with prior art automated video quality 
assessment techniques. 

Figure 14 illustrates a second embodiment of the present invention, being one 
that is implemented In software. Here, the various processes provided by the present 
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invention are performed by a computer 140, wliich Is arranged to receive both the 
reference video sequence, and the test video sequence. The computer 140 is provided 
with a program storage device such as a hard dlsl< drive, writable CD or DVD. memory or 
the like, in which are stored various computer programs which contain instructions which 
when executed by the computer 140 cause the computer to perform the present 
Invention. More particulariy. a control and interface program 148 is provided which when 
run allows a user of the computer to control the computer to begin to process test and 
reference video sequences in accordance with the invention, and to allow the computer to 
output the test results to the user on the screen. Such a control and Interface program is 
preferably graphically based, and the an-angement of such a program would be apparent 
to one skilled in the art. 

Additionally provided is an Integrator program 144. a control and Interface 
program 148. a texture analyser program 150, a matching program 152. a cropping 
program 154. a spatial frequency analysis program 156, a signal to noise ratio calculation 
program 158. and an edge analyser prograiji 160. The operations of each of these 
programs will be briefly discussed below in the context of an example operation. 

Imagine that a user of the general purpose computer 140 wishes to test a video 
sequence. The user first of all starts the control and Interface program 148. which 
provides controls to the user to allow the user to specify the test and reference video 
sequences to be processed. Once the user has instructed the computer 140 to. 
commence processing via the control and interface program 148. the control and 
interface program 148 then causes the computer 140 to commence processing, by 
causing various of the other programs to be excuted and to process data as appropriate. 
Therefore, upon receiving instructions from the user to commence processing the control 
and interface program 148 firstly causes the cropping program 154 to execute, virtiich acts 
to crop the input reference and test video sequences in exactly the same manner as 
already described in respect of the crop and offset module 32. That is. the cropping 
program 145 operates to crop the input video sequences in the same manner as 
previously described in respect of Figure 3. The thus cropped video sequences are then, 
stored in a wori<ing data area 146 of the data storage device 142. 

Following cropping of the input video sequences, next the control and interface 
program 148 causes the matching program 152 to mn. The matching program 152 
accesses the cropped reference and test video sequences in the working data portion 
146 of the storage device 142, and acts to perform sequence matching in exactly the 
same manner as the sequence matcher module 30 as previously described with respect 
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to Figures 4 and 5. That Is. the matching program 152 operates according to the flow 
diagram of Figure 5 to produce the. matched reference sequence, which is then stored 
within the worthing data portion 146 of the data storage device 142. 

iHaving generated the matched reference sequence, the controi and interface 
program 148 then launches the spatial frequency analysis program 156, which operates 
to perform spatial frequency analysis on the matched reference fields/frames and the test 
fields/frames in exactly the same manner as the spatial frequency analyser 22 as 
previously described, and therefore the spatial frequency analysis program 156 operates 
to perfomi the flow diagrams of both Figures 8 and 9 so as to generate the PySNR video 
characteristic values, which are then stored In the working data portion 146 of the storage 
device 142. 

Next, the control and interface program 148 launches the signal to noise ratio 
calculation program 158, which then accesses the test-wdeo fields/frames and the 
matched reference video fields/frames within the working data portion 146, and calculates 
luminance and chrominance signal to noise ratios in the previous manner as described. 
That is, the signal to noise ratio calculation program 158 operates in an identical manner 
to the luminance and chrominance peak signal to noise ratio analyser 24 as previously 
described. The resultant luminance and chrominance signal to noise ratios calculated by 
the signal to noise ratio calculation program 158 are stored in the working data area 146 
of the storage device. 

The control and interface program 148 then launches the texture analyser 
program 150. The texture analyser program 150 then accesses the test video 
fields/frames from the working data area 146, and acts to calculate the TextureDeg 
parameter in the same manner as previously described in respect of the texture analyser 
28. Therefore, the texture analyser program 150 operates In accordance with the flow 
diagram of Figure 12. If required the texture analyser program may also calculate 
TextureRef and Texture Mref parameters in addition, and in the same way. 

Following the operation of the texture analyser program 150, the control and 
Interface program 148 then launches the edge analyser program 160. The edge analyser 
program acts to access the matched reference fields/frames within the working data area 
146 of the storage device 142, as well as the test video fields/frames. Then, the edge 
analyser program 160 operates substantially as described previously in respect of the 
edge analyser 26, that is the edge analyser program 160 substantially performs the steps 
set out in the flow diagram of Figure 11. The output of the edge analyser program is the 
ED/f parameter, which is stored in the working area 146. 
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At this stage In the operation of the second embodiment, each of the analyser 
programs has been executed, and the working area 146 therefore contains all of the 
video characteristic parameter values which may be used as Inputs for an integrator. 
Therefore, the next operation is that the control and interface program 148 launches the 
integrator program 144. which acts to access the working area 146 to read the 
characteristic parameter values therefrom as appropriate, and to integrate the parameters 
together in order to generate the final video quality value PDMOS. The operation of the 
integrator program 144 is substantially identical to that of the integrator stage 4 as 
previously described. Therefore, the Integrator program 144 operates substantially in 
accordance with the flow diagram of Figure 13. as previously described! 

The second embodiment of the invention therefore provides a software 
embodiment, which acts to generate the video quality value in substantially the same 
manner as previously described in respect of the first embodiment. It should be noted 
here, however, that the first embodiment as described may also be Implemented In 
software or altematively may be implemented with hardware elements or a mixture of 
both software and hardware. In this respect, the first embodiment should be considered 
to be a more general embodiment than the second embodiment. 

Various other modifications may be made to the described embodiments to 
provide further embodiments. For example, in a further embodiment the matcliing 
process may be performed using different block sizes. In particular, an em)r minimisation 
block size greater than the reconstructed block size would give error minimisation for 
over-lapping windows, with some pixels being considered for the matching of more than 
one degraded block. Within such a process the matching module would still act to split 
the test video field/frame up into the same number of blocks as previously described, but 
vyouid then define a matching element which wholly contained the pixels of the block to 
be matched, but which also contained further pixels surrounding the block to be matched. 
This matching element would then be used in the en-or minimisation function to find a 
matching block within the reference sequence to the block to be matched. This may give 
benefits by increasing the accuracy of the matching. 

The matching process used in the present invention above can overcome the 
effects of a number of different types of distortion, from consistent and regular field 
misalignments through to warping and more transient block-based degradations. 
Differences between the sensitivity of PSNR and matched PSNR measures to various 
distortions are now presented to illustrate the benefits of the matching process. All 
presented results are for luminance intensity pixel values only. 
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Dealing first with spatial misalignment artefacts, spatial misalignment between 
the reference and degraded fields, by just one pixel, can have a significant impact on 
SNR measures whilst being imperceptible to a viewer. The matching process provided 
by the invention can handle both consistent and time-varying spatial offsets up to the 
perceptual-based search limits set within the algorithm. 

Figure 15 shows PSNR results calculated from a 625 sequence that has been 
MPEG2 encoded to 3 Mb/s and the reference and degraded sequences spatially (and 
temporally) aligned. The PSNR plot shows basic PSNR calculated between each field of 
the reference and degraded according to: 



PSA«(«) = 101og,o(X*r«255V(2 E(/-e/(«./,J)-deg(«,/.y))^)) 

(15-1) 

where X and Y are the field pixel dimensions. 

The MPSNR plot shows results when the matched reference sequence replaces 
the reference sequence as an input to the SNR module. In this case, close alignment 
can be seen between the PSNR and MPSNR results. Figure 16 shows the effect of a 2 

15 pixel horizontal offset on the degraded sequence. It can be seen that MPSNR is 
unchanged whereas PSNR is reduced by up to 7 dB. 

Figure 17 shows the effects of a random horizontal field-jitter between +/- 4 
pixels for the same test sequences. Again, it can be seen that the matched PSNR is 
unaffected by the spatial jitter, whilst the normal PSNR is greatly changed. 

20 With regards to temporal misalignments, temporal misalignment of the reference 

and degraded sequences can also have a significant effect on the SNR measures. In this 
case the severity of the effects depends on the time-varying properties of the video 
sequence. As with spatial misalignment, the matching process can handle both constant 
and time-varying temporal offsets within the perceptual-based search limits of the 

25 algorithm. 

An interesting example of the benefits of the matching process provided by the 
invention Is shown in Figure 18, where a 625 broadcast sequence has been H.263 
encoded at CIF resolution with the result that successive degraded fields are identical. 
The PSNR plot shows greatly reduced values for alternate fields due to the reduced 
30 resolution encoding, whereas the MPSNR is unaffected. - 

The matching process can also handle irregular temporal misalignments and 
Figure 19 illustrates this for the random freezing of fields. In this example, each field has 
a 10% chance of being frozen (replacing the next field in time). 
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Finally, with respect to bloclc-level distortions, the matching process of the 
Invention Is designed to handle distortions that are more complex than the field-based 
spatial and temporal misalignments considered so far. The matching of blocks that are 
small relative to the overall size of the field enables the undesirable effects of distortions 
such as image warping, missing lines and block freezing to be reduced. 

Figure 20 shows the effect that "block-freezing" has on the PSNR and MPSNR 
measures. The "block-fl-eezing" simulation randomly selects only 85% of 8x8 pixel blocks 
within a field to be updated and in this example the process was not started until after the 
25*^ field. Whilst tests showed the distortion to be Imperceptible, It can be seen that the 
PSNR is considerably reduced. However, as the matching process is able to track the 
frozen blocks, the MPSNR measure is barely affected. 

Unless the context cleariy requires othenwise, throughout the description and the 
claims, the words "comprise", "comprising" and the like are to be construed in an 
inclusive as opposed to an exclusive or exhaustive sense; that is to say. in the sense of 
"including, but not limited to". 

Moreover, for the avoidance of doubt, where reference has been given to a prior 
art document or disclosure, whose contents, whether as a whole or in part thereof, are 
necessary for the understanding of the operation or implementation of any of the 
embodiments of the present invention by the intended reader, being a man skilled in the 
art. then said contents should be taken as being incorporated herein by said reference 
thereto. 
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CLAIMS 

1 . A video quality assessment method, comprising tiie steps of: 

matching sub-fleld/frame elements of a test video field/frame with con-esponding 
5 sub-field/frame elements of at least one reference video field/frame; and 

generating a video quality value in dependence on the matched sub-field/frame 
elements of the test and reference video fields/frames. 



2. A method according to claim 1 , wherein the matching step further comprises, for 
10 a sub-field/frame element of the test video field/frame, searching for a matching sub- 
field/frame element within M1 preceding and/or M2 succeeding reference video 
fields/frames to a temporally con-esponding reference video field/frame to the test video 
field/frame. 

15 3. A method according to claim 2, wherein M1 and M2 are predefined. 

4. A method according to claims 2 or 3, wherein the searching step further 
comprises searching within a spatially bounded region of the reference video 
fields/frames about the con-esponding position within the reference fields/frames as the 

20 test sub-field/frame element takes within the test video field/frame. 

5. A method according to claim 4, wherein the spatial extent of the search region is 
predefined. 

25 6. A method according to any of the preceding claims, wherein the matching step 
further comprises, for a sub-field/frame element of the test video field/frame: 

defining a matching template comprising a portion of the test video field/frame 
including the sub-field/frame element; and 

using the defined matching template to search for matching sub-field/frame 
30 elements in the at least one reference video field/frame. 

7. A method according to any of the preceding claims, wherein the matching step 
further comprises calculating one or more matching statistic values and/or matching 
vectors; and wherein the generating step generates the video quality parameter in further 
35 dependence on the calculated matching statistic values and/or matching vectors. 
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8. A method according to claim 7. wherein the calculating step comprises- 
constructing one or more histograms relating to the searched area(s) of the 

reference video field(s)/frame(s); and 

calculating a matching statistic value for each histogram relating to the 
proportion of matched elements which contribute to the peak of the histogram. 

9. A method according to any of the preceding claims, wherein the generating step 
further comprises: 

calculating a plurality of video characteristic values respectively relating to 
characteristics of the test and/or reference video fields/frames In dependence on the 
matched sub-fleld/frame elements of the test and reference video fields/frames- and 

integrating at least the calculated video characteristic values together to give the 
video quality value. 
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A method according to claim 9 when dependent on claims 7 or 8. wherein the 
integrating step further includes integrating the matching statistic value(s) with the 
calculated video characteristic values to give the video quality value. 

11- A method according to claims. 9 or 10. wherein the video characteristic values 
are respectively any two or more of the following values: one or more spatial frequency 
values; one or more texture values; at least one edge value; at least one luminance 
signal to noise ratio value; and/or one or more chrominance signal to noise ratio values. 

25 12. A method according to claim 11. wherein the calculation of the edge value 
comprises, for a test field/frame: 

counting a number of edges in each sub-fleld/frame element of the test 
field/frame; 

counting a number of edges In each sub-field/frame element of the at least one 
30 reference field/frame matched to the sub-fleid/frame elements of the test field/frame- and 
detemiining an edge value for the test field/frame In dependence on the 
respective counts. 
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A method according to claim 12. wherein the detemilning step further comprises- 
calculating difference values between each pair of respective counts; 
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putting each calculated difference value to tlie power Q ; 
summing the resulting values to give a sum value; and 
putting the sum value to the power 1/Q to give the edge value. 



5 14. A method according to any of claims 9 to 13, wherein the integrating step further 
comprises weighting each value by a predetennined weighting factor; and summing the 
weighted values to give the video quality value. 

15. A method according to claim 1 4, wherein the summing step is further an^nged to 
10 sum the weighted values with a predetennined offset value. 

16. A method according to any of claims 14 or 15, wherein the weighting factors and 
the offset value are dependent on the type of the test and reference video fields/frames. 

15 17. A computer program or suite of programs so an-anged such that when executed 
by a computer system it/they cause/s the system to perfonn the method of any of claims 
1 to 16. 

18. A modulated canier signal Incorporating data conresponding to the computer 
20 program or at least one of the suite of programs of claim 17. 

19. , A computer readable storage medium storing the computer program or at least 
one of the suite of computer programs of claim 17. 

25 20. A system for video quality assessment, comprising: 

matching means for matching sub-field/frame elements of a test video field/frame 
with conresponding sub-field/frame elements of at least one reference video field/frame; 
and 

video processing means arranged in use to generate a video quality value in 
30 dependence on the matched sub-field/frame elements of the test and reference video 
fields/frames. 



21 . A system according to claim 20, wherein the matching means further comprises, 
temporal searching means arranged in use to search for a matching sub-field/frame 



41 

element within M1 preceding and/or M2 succeeding reference video fields/frames to a 
temporally corresponding reference video field/frame to the test video field/frame. 

22. A system according to claim 21, wherein Ml and M2 are predefined. 

5 

23. A system according to claims 21 or 22, and further comprising spatial searching 
means arranged in use to search within a spatially bounded region of the reference video 
fields/frames about the corresponding position within the reference fields/frames as the 
test sub-field/frame element takes within the test video field/frame. 

10 

24. A system according to claim 23, wherein the spatial extent of the search region 
is predefined. 

25. A system according to any of claims 20 to 24, wherein the matching means 
1 5 further comprises:- 

means for defining a matching template comprising a portion of the test video 
field/frame Including the sub-field/frame element; and 

means for using the defined matching template to search for matching sub- 
field/frame elements in the at least one reference video field/frame. 

20 

26. A system according to any of claims 20 to 25, wherein the matching means 
further comprises calculating means arranged in use to calculate One or more matching 
statistic values and/or matching vectors; and wherein the video processing means is 
further arranged in use to generate the video quality parameter in further dependence on 

25 the calculated matching statistic values and/or matching vectors. 

27. A system according to claim 26, wherein the calculating means further 
comprises: 

histogram constructing means arranged in use to construct one or more 
30 histograms relating to the searched area(s) of the reference video field(s)/frame(s); and 

matching statistic calculating means for calculating a matching statistic value for 
each histogram relating to the proportion of matched elements which contribute to the 
peak of the histogram. 
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28. A system according to any of claims 20 to 27, wherein the video processing 
means further comprises: 

a plurality of analysis means respectively arranged in use to calculate a plurality 
of video characteristic values respectively relating to characteristics of the test and/or 
5 reference video fields/frames in dependence on the matched sub-field/frame elements of 
the test and reference video fields/frames; and 

an integration means for integrating at least the calculated video characteristic 
values together to give the video quality value. 

10 29. A system according to claim 28 when dependent on claims 26 or 27. wherein the 
integration means is further arranged to integrate the matching statistic value(s) with the 
calculated video characteristic values to give the video quality value. 

30. A system according to claims 28 or 29, wherein the video characteristic values 
15 are respectively any two or more of the following values: one or more spatial frequency 

values; one or more texture values; at least one edge value; at least one luminance 
signal to noise ratio value; and/or one or more chrominance signal to noise ratio values. 

31. A system according to claim 30, and further comprising edge calculation means 
20 comprising: 

means for counting a number of edges in each sub-field/frame element of the 
test field/frame; 

means for counting a number of edges in each sub-field/frame element of the at 
least one reference field/frame matched to the sub-field/frame elements of the test 
25 field/frame; and 

means for determining an edge value for the test field/frame in dependence on 
the respective counts. 

32. A system according to claim 31, wherein the means for determining further 
30 comprises an arithmetic calculator means arranged in use to: 

calculate difference values between each pair of respective counts; 

put each calculated difference value to the power Q ; 

sum the resulting values to give a sum value; and 

put the sum value to the power VQ to give the edge value. 
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33. A system according to any of claims 28 to 32. wherein the integrating means 
further comprises weighting means for weighting each value by a predetermined 
weighting factor; and summing means for summing the weighted values to give the video 
quality value. 

5 

34. A system according to claim 33. wherein the summing means Is further arranged 
to sum the weighted values with a predetermined offset value. 

35. A system according to any of claims 33 or 34. wherein the weighting factors and 
10 the offset value are dependent on the type of the test and reference video fields/frames. 



15 
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ABSTRACT 

Method and Sy stem for Video Quality Assessment 
A method and system for automated video quality assessment which reduces the 
adverse effects of sub-field/frame misalignments between the reference and test 
sequences. More particularly, the invention provides for misalignments down to a sub- 
field/frame level to be handled by individually matching sub-field/frame elements of a test 
video field/frame with sub-field/frame elements from a reference video field/frame. The 
use of a matching element size that is significantly smaller than the video field/frame size 
enables transient sub-field/frame misalignments to be effectively tracked. 



Figure (2) 
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